Tractable approximations for probabilistic models: the adaptive Thouless-Anderson-Palmer mean field approach.
نویسندگان
چکیده
We develop an advanced mean field method for approximating averages in probabilistic data models that is based on the Thouless-Anderson-Palmer (TAP) approach of disorder physics. In contrast to conventional TAP, where the knowledge of the distribution of couplings between the random variables is required, our method adapts to the concrete couplings. We demonstrate the validity of our approach, which is so far restricted to models with nonglassy behavior, by replica calculations for a wide class of models as well as by simulations for a real data set.
منابع مشابه
Adaptive and self-averaging Thouless-Anderson-Palmer mean-field theory for probabilistic modeling.
We develop a generalization of the Thouless-Anderson-Palmer (TAP) mean-field approach of disorder physics, which makes the method applicable to the computation of approximate averages in probabilistic models for real data. In contrast to the conventional TAP approach, where the knowledge of the distribution of couplings between the random variables is required, our method adapts to the concrete...
متن کاملPerturbative Interpretation of Adaptive Thouless-Anderson-Palmer Free Energy
In conventional well-known derivation methods for the adaptive Thouless-Anderson-Palmer (TAP) free energy, special assumptions that are difficult to mathematically justify except in some mean-field models, must be made. Here, we present a new adaptive TAP free energy derivation method. Using this derivation technique, without any special assumptions, the adaptive TAP free energy can be simply o...
متن کاملTraining Restricted Boltzmann Machine via the Thouless-Anderson-Palmer free energy
Restricted Boltzmann machines are undirected neural networks which have been shown to be effective in many applications, including serving as initializations for training deep multi-layer neural networks. One of the main reasons for their success is the existence of efficient and practical stochastic algorithms, such as contrastive divergence, for unsupervised training. We propose an alternativ...
متن کاملAdaptive Tap Equations
We develop a TAP mean eld approach to models with quadratic interactions which does not assume a speciic randomness of the couplings but rather adapts to the concrete data. The method is based on an extra set of mean eld equations for the Onsager correction term to the naive mean eld result. We present applications for the Hoppeld model and for a Bayesian classiier. 1.1 Introduction Mean eld (M...
متن کاملInformation Geometry of Mean-Field Approximation
I present a general theory of mean-field approximation based on information geometry and applicable not only to Boltzmann machines but also to wider classes of statistical models. Using perturbation expansion of the Kullback divergence (or Plefka expansion in statistical physics), a formulation of mean-field approximation of general orders is derived. It includes in a natural way the "naive" me...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- Physical review letters
دوره 86 17 شماره
صفحات -
تاریخ انتشار 2001